1. Field of the Invention
The present invention generally relates to video processing and in particular, to H.264 and Moving Picture Experts Group (MPEG)-4 Frame Rate Conversion (FRC) using motion interpolation that uses motion estimation and motion compensation.
2. Description of the Related Art
Frame Rate Conversion (FRC) refers to converting a number of frames output per second, and generally to a technique for converting the frame rate of a video sequence that has been compressed at a low frame rate into a high frame rate. FRC is usually applied when a video transmission signal and a display have different frame rates and a Personal Computer (PC), a High-Definition Television (HDTV), and the like exchange programs in various signal formats.
In the early stage of FRC, various methods for converting the frame rate of an uncompressed video signal were developed and used. However, as video compression methods of the SIO/IEC JTC1 group or the ITU-T group have been developed with the sharp increase in the volume of video data, FRC is now also applied to a compressed video signal and a video system using the compressed video signal during video compression and transmission.
FIG. 1 illustrates the application of an FRC apparatus 102. Referring to FIG. 1, a video source 101 may be that of a mobile device such as a Digital Multimedia Broadcasting (DMB) device, a Personal Multimedia Player (PMP), and the like. The frame rate of the video source 101 is converted by the FRC apparatus 102 into a video source suitable for display on a display device 103.
FIG. 2 is a block diagram of a conventional FRC apparatus. Referring to FIG. 2, the FRC apparatus acquires a motion vector through block-based motion estimation and generates a new frame, i.e., an interpolated frame, through motion compensation using the motion vector. An image segmentation unit 110 segments an image of a single frame into several regions for efficient motion estimation. To this end, two stages are performed: the first one is to segment the entire frame into a changed region and an unchanged region and the second one is to segment the unchanged region into a covered region, an uncovered region, a background region, and an object region.
A motion estimation unit 120 performs motion estimation for each block using a Block Matching Algorithm (BMA). A vector smoothing unit 130 smoothes a motion vector acquired in the previous stage. A Motion Compensated Interpolation (MCI) unit 140 then completes an interpolated frame through MCI.
FIGS. 3A and 3B schematically illustrate conventional MCI using a bidirectional motion vector. More specifically, FIG. 3A illustrates an operation of setting an initial value of a bidirectional motion vector and FIG. 3B illustrates an operation of performing MCI. It should be noted that a previous frame, an interpolated frame, and a current frame illustrated in FIG. 3A are expressed along an axis that is perpendicular or parallel to a time axis.
Conventional MCI first performs motion estimation and then generates an interpolated frame using a unidirectional or bidirectional motion vector between a previous frame and a current frame. In FIGS. 3A and 3B, a bidirectional motion vector is used in order to prevent the generation of an overlap and a hole that may occur during interpolation using a forward motion vector between frames that precede and follow an interpolated frame. A block grid of an interpolated frame is appropriately set using an initial bidirectional motion vector in FIG. 3A, and motion estimation and MCI are performed with respect to the interpolated frame using a BMA for each block in FIG. 3B.
However, motion estimation and MCI using a bidirectional motion vector require a large amount of computational effort. Such computational effort is limited in mobile devices. To solve the problem of performing motion estimation and MCI in mobile devices, a method in which motion estimation is skipped and motion information is input from a decoder is under development. However, the use of MCI still requires a large amount of computation effort.